The Terra Galleria Blog – QT Luong

Being a large format photographer, I tend to be a bit picky regarding image quality. When using 35mm cameras, since one needs to enlarge so much from the small image capture, everything has to be the best possible in order to yield a sharp print.

Of course, image quality is a minor aspect of photography, compared to all the other factors that go into making great photographs. However, to quote Norman Koren, “the technique you develop in testing cameras and lenses will spill over to your daily photography. Alfred Stieglitz tested film and developers extensively when he discovered photography as a student in Berlin. Ansel Adams performed extensive tests in the development of his zone system. Although nobody would claim that testing is responsible for their unique vision, it certainly contributed to the skill that transformed their vision into prints of transcendent beauty.”

Back in the early 1990s, I was doing quite a bit of lens testing in order to understand which lenses were sharper, and how the choice of focal lengths and f-stops affected image quality. One photographed targets (that had to be carefully labeled to identify the parameters at a later stage), had the film processed, and then squinted with a microscope, struggling to determine which groups of bars were “resolved”. This approach can yield very reliable measurements, including resolution in lpm (the best example of them today is found on William Castleman’s EOS lens reviews) but it was tedious, and I eventually lost interest.

This changed in 2004, when Norman Koren released his first version of Imatest, a software to automate digital image quality measurement. You still need to set up the target shots carefully, but now you just input sets of digital images into the software, and then read measurements. Compared to other solutions, Imatest distinguishes itself by the simplicity of the test targets it uses, consisting of two uniform regions separated by a straight border. Those targets are easy and inexpensive to make.

In late 2005, Canon released the 24-105/4, which was anything I wished for in an all-purpose lens. The only question mark was image quality. To find out for myself, in early 2006, I tested this lens against my 24-70/2.8 and other EOS lenses on a Canon 1Ds mark II. I have previously written a comparative review of the 24-105/4 and 24-70/2.8 lenses. This post summarizes my findings with respect to sharpness.

I won’t detail my experimental procedure here (this may be the subject of a future post) but let me say that tests were done very, very carefully and the results were repeatable. Imatest can provide a lot of information, including, in the current version, 3D graphs and a MTF graph similar to the ones published by Canon in EF Lens Work. This test was conducted with an older version of Imatest. For the sake of simplicity, I am using here a single number to represent image quality. This number is the MTF 50 calculated by Imatest, integrating several measurements over the image area. The larger the number, the higher the resolution/contrast.

Lenses tested:
24-70: Canon EF 24-70mm f/2.8L USM
24-105: three copies of Canon EF 24-105mm f/4L IS USM
12-24: Sigma 12-24mm f/4.5-5.6 EX DG IF HSM
17-40: Canon EF 17-40mm f/4L USM
70-300: Canon EF 70-300mm f/4-5.6 IS USM
70-200: Canon EF 70-200mm f/2.8 IS USM
24TSE: Canon TS-E 24mm f/3.5L
45TSE: Canon TS-E 45mm f/2.8
50: Canon EF 50mm f1.4 USM
85: Canon EF 85mm f/1.8 USM
90TSE: Canon TS-E 90mm f/2.8
100: Canon EF 100mm f/2.8 Macro USM

@24	24-70	24-105  	17-40	12-24		24TSE
2.8	26.2	na/na/na        na      na		na			      		    
4	29.8	28.2/34.8/35.0	28.3	na     		27  
5.6	31.6	33.6/38.4/35.8	33.3	23.6 		29.8
8	31.7	35.7/37.4/34.6	36.1	29   		31.1
11	30.2	33.2/33.4/32.3	34.6	29   		30.3
16	27.2	29.0/29.1/28.9	30.3	26.7 		26.9

@35	24-70	24-105		17-40
2.8	28.2    na/na/na        na
4	32.3	32.6/36.0/34.0	21.8
5.6	33.7	37.4/38.6/37.5	23.2
8	33.4	36.8/37.1/36.6	29.7
11	31.6	34.3/34.6/34.1	34.3
16	28	29.6/30.0/30.3	30

@50	24-70	24-105				50	45TSE
1.4	na      na				13.7    na
2	na	na				21.4    na
2.8	24	na				32.87	33
4	29.2	24.4/32.7/32.5			40.11	35.6
5.6	31.5	35.8/33.9/37.1			40.7	35.4
8	32	37.2/33.3/35.1			39.7	34.6
11	31.2	35.3/34.2/34.5			36.2	33.2
16	28.3	19.7/31.2/31.4			31.9	30

@70	24-70	24-105		70-300	70-200	85	90TSE
2	na      na		na      na	25.2	na
2.8	20.3	na		na	29.9	28.8	29
4	23.66	25.2/25.1/26.5	27.9	32.2	32.6	34.2
5.6	26.98	32.3/37.6/35.5	32.8	31.7	36	38.2
8	28.77	28.8/36.9/34.6	37.1	31.2	37.2	38.6
11	28.4	34.0/35.4/33.6	35.3	29.8	35.1	36
16	26.52	30.3/31.0/30.1	31.2	28	30.8	31.4

@105		24-105		70-300	70-200	100-400	100
2.8		na		na	32.6	na      29.8	
4		20.1/23.1/26.0	na	34.2	na      34.1	
5.6		26.0/33.3/34.9	34.3	34.1	28	34.6	
8		23.5/35.8/35.3	37.2	33	30.6	36	
11		18.8/34.4/33.3	36.4	31.2	31	35.4	
16		29.4/30.7/30.1	32.2	28.7	28.6	30.5
	

A wealth of information can be found in the table above, but here are a few general observations. Some will be well-known to some readers, but are worth repeating for others.

• There is a significant performance variation between copies of the same lens. In general, Japanese lenses are not individually tested (unlike some German lenses, but they cost three times more). If you want to be sure to get a “good” copy of a lens, buy a few, test them, and keep the best.
  Amazon has free shipping both ways, and an easy 1 month return period, but beware that this does not apply to third-party sellers, which have their own policies. Rather than buying from a third party seller on Amazon, I prefer B&H. They have a 14-day return period and free shipping for some items only, however I have confirmed with their customer service that there will not be a problem to return lenses ordered for testing. Other vendors may charge restocking fees.
• All lenses are sharpest at middle apertures (sharpness is limited by abberations at wide apertures and by diffraction at smaller apertures. In fact, by f16, all lenses perform almost the same, which is why I didn’t bother to make measurements at f22). If given the choice, use a middle aperture such as f8.
• Zoom lenses can be sharper than fixed focal lenses.
• Inexpensive lenses can be sharper than more expensive lenses.
• Zoom lenses are in general weakest at longest focal. If you have the choice between two overlapping zooms, use the one which is less zoomed out.

Key findings about the 24-105:

• 24-105 is sharper than 24-70 at all focal lengths and apertures. This comes as a surprise to some, as the 24-70 is the more “professional-looking” lens, with its 2.8 aperture, better construction and hood, and slightly higher price, but the conclusion is consistent with Chasseur d’Images tests.
• 24-105 delivers excellent sharpness wide-open at wide focal lengths, but needs to be stopped down 1 f-stop at longer focal lenghts

Findings about other lenses:

• The sharpest lens by far, the 50/1.4 is also the least expensive. Excellent lenses in this focal length are very easy to design and manufacture. There is a reason why those “normal” lenses were, in the past, bundled with SLR cameras bodies.
• Of my three tele-zoom lenses, the sharpest one is also the least expensive, the 70-300, which in particular is sharper than the much-praised 70-200/2.8.
• Of the three TSE lenses, the sharpest is the 90, followed by the 45, and then the 24. This is expected, due to the difficulty of designing a short lens with a large image circle.